feat: model and serializer for audio tracks by ceberam · Pull Request #426 · docling-project/docling-core

ceberam · 2025-11-17T08:34:36Z

This PR introduces a new type of provenance object for media files.

ProvenanceTrack for media tracks
Refactoring of other classes for provenance type check
Utility classes and methods in a new module on WebVTT for reuse
For testing the integration with docling, check the draft PR feat: webvtt and source tracker docling#2787

It also addresses docling-project/docling#2525

github-actions · 2025-11-17T08:34:45Z

✅ DCO Check Passed

Thanks @ceberam, all your commits are properly signed off. 🎉

mergify · 2025-11-17T08:35:10Z

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

🟢 Require two reviewer for test updates

Wonderful, this rule succeeded.

When test data is updated, we require two reviewers

#approved-reviews-by >= 2

codecov · 2025-11-17T08:45:35Z

Codecov Report

❌ Patch coverage is 90.01721% with 58 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling_core/transforms/serializer/webvtt.py	82.01%	34 Missing ⚠️
docling_core/types/doc/webvtt.py	93.16%	24 Missing ⚠️

📢 Thoughts on this report? Let us know!

dosubot · 2025-12-15T08:40:03Z

Related Documentation

Checked 7 published document(s) in 0 knowledge base(s). No updates required.

^{How did I do? Any feedback?}

PeterStaar-IBM

Nice!

vagenas

Align new union naming to differentiate from "legacy" prov (e.g. call it SourceType?), also aligned with new field
Unroll the tags field into the individual fields we consider relevant for now, e.g. "voice" / "id" (I don't think we need "classes" now, as it's quite VTT-specific). The fact that we would be losing some information on VTT is actually consistent with other import/export paths (e.g. HTML with embedded CSS).
validate upon assignment could be considered outside this PR
DoclingDocument.add_text() can be extended with a new optional param for source

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

…le types Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Since WebVTTTimestamp is used in DoclingDocument, the class should be public. Strengthen validation of cue language start tag annotation. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Add a DoclingDocument serializer to WebVTT format. Improve WebVTT data model. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Add 'text/vtt' as extra MIME type to support WebVTT serialization, since it is not supported by 'mimetypes' with python < 3.11 Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Classes and fields that are related to the new source type should aign with their names. The term 'provenance' will identify the legacy implementation. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

Drop the validation on field assignment in NodeItem objects. Add the 'source' argument in the convenient function 'add_text' to create TextItem with track source data. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com> refactor(webvtt): drop cue span classes, 'lang' and 'c' tags Drop WebVTT formatting features not covered by Docling across formats. Only 'u', 'b', 'i', and 'v' are supported and without classes. Make 'v' tag explicit as 'voice' feature in SourceTrack class. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

ceberam · 2026-01-26T13:11:04Z

Thanks @vagenas for the review.
I have pushed some commits addressing the points you raised.

Align new union naming to differentiate from "legacy" prov (e.g. call it SourceType?), also aligned with new field

OK ✅

2. Unroll the `tags` field into the individual fields we consider relevant for now, e.g. "voice" / "id" (I don't think we need "classes" now, as it's quite VTT-specific). The fact that we would be losing some information on VTT is actually consistent with other import/export paths (e.g. HTML with embedded CSS).

⚠️ OK, but we need to be aware that we are no longer supporting all the WebVTT cue span features anymore and therefore the cycle

WebVTT file --> (WebVTTDocumentBackend) --> DoclingDocument -> (WebVTTDocSerializer) --> WebVTT file

may now imply loss of information, even though the resulting file will still be a valid WebVTT according to the specs.

3. validate upon assignment could be considered outside this PR

⚠️ OK, but let's see later on if we want to activate it as a safeguard. I have seen in docling-core some patterns of field assignments after instantiating a model.

4. `DoclingDocument.add_text()` can be extended with a new optional param for `source`

OK ✅

cau-git

LGTM

ceberam force-pushed the dev/webvtt branch 2 times, most recently from 295c5ac to 0c201e2 Compare November 17, 2025 08:43

ceberam force-pushed the dev/webvtt branch from 0c201e2 to 740dadb Compare November 27, 2025 18:14

ceberam self-assigned this Nov 27, 2025

ceberam force-pushed the dev/webvtt branch 2 times, most recently from 9e26200 to a0f4404 Compare December 4, 2025 10:02

ceberam added the enhancement New feature or request label Dec 4, 2025

ceberam force-pushed the dev/webvtt branch 3 times, most recently from c2efd15 to 4ecde67 Compare December 5, 2025 14:20

ceberam force-pushed the dev/webvtt branch from 4ecde67 to 3a49623 Compare December 12, 2025 10:32

ceberam mentioned this pull request Dec 15, 2025

feat: webvtt and source tracker docling-project/docling#2787

Merged

3 tasks

ceberam marked this pull request as ready for review December 15, 2025 08:39

ceberam force-pushed the dev/webvtt branch 2 times, most recently from c56d14a to c56a691 Compare January 6, 2026 15:15

PeterStaar-IBM requested review from PeterStaar-IBM, cau-git, dolfim-ibm and vagenas January 19, 2026 10:22

ceberam force-pushed the dev/webvtt branch 4 times, most recently from 19a2dbe to 5e0a787 Compare January 23, 2026 15:26

PeterStaar-IBM previously approved these changes Jan 26, 2026

View reviewed changes

vagenas reviewed Jan 26, 2026

View reviewed changes

refactor: move WebVTT data model from docling

c4bcf78

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

ceberam added 21 commits January 26, 2026 13:10

fix(webvtt): deal with HTML entities in cue text spans

813d2b5

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): support more WebVTT models

67305d5

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(DoclingDocument): create a new provenance model for media fi…

a9ff665

…le types Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): make WebVTTTimestamp public

110c319

Since WebVTTTimestamp is used in DoclingDocument, the class should be public. Strengthen validation of cue language start tag annotation. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): set languages to a list of strings in ProvenanceTrack

190fd27

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

tests(webvtt): add test for ProvenanceTrack

0ee98a0

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): make all WebVTT classes public for reuse

5206c0c

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

chore(webvtt): preserve newlines as WebVTTLineTerminator

15ec4c4

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): set ProvenanceTrack time fields as float

714c60a

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

chore(webvtt): ensure start time offsets are in sequence

612bcda

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

chore(webvtt): improve regex to remove note,region,style blocks

10cdcdb

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

chore(webvtt): parse the WebVTT file title

b0b5c53

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

chore(webvtt): rebase to latest changes in idoctags

4cf5e8e

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

feat(webvtt): add WebVTT serializer

993195e

Add a DoclingDocument serializer to WebVTT format. Improve WebVTT data model. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

fix(webvtt): add 'text/vtt' as extra mimetype

c2f6350

Add 'text/vtt' as extra MIME type to support WebVTT serialization, since it is not supported by 'mimetypes' with python < 3.11 Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): roll back DocItem.prov as list of ProvenanceItem

fafdf3f

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

tests(webvtt): fix test with STYLE and NOTE blocks

685a611

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

style(webvtt): apply X | Y annotation instead of Optional, Union

50f8ba9

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): simplify TrackProvenance model with tags

aedca13

Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

refactor(webvtt): align class and field names to new 'source' type

563c041

Classes and fields that are related to the new source type should aign with their names. The term 'provenance' will identify the legacy implementation. Signed-off-by: Cesar Berrospi Ramis <ceb@zurich.ibm.com>

ceberam dismissed PeterStaar-IBM’s stale review via 621e55e January 26, 2026 13:02

ceberam force-pushed the dev/webvtt branch from 55fb835 to 621e55e Compare January 26, 2026 13:02

vagenas approved these changes Jan 26, 2026

View reviewed changes

cau-git approved these changes Jan 27, 2026

View reviewed changes

PeterStaar-IBM approved these changes Jan 27, 2026

View reviewed changes

ceberam merged commit c8f3c01 into main Jan 27, 2026
12 checks passed

ceberam deleted the dev/webvtt branch January 27, 2026 13:45

ceberam mentioned this pull request Feb 25, 2026

Unsupported features in WebVTT parser docling-project/docling#2525

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: model and serializer for audio tracks#426

feat: model and serializer for audio tracks#426
ceberam merged 22 commits intomainfrom
dev/webvtt

ceberam commented Nov 17, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Nov 17, 2025 •

edited

Loading

Uh oh!

mergify bot commented Nov 17, 2025 •

edited

Loading

Uh oh!

codecov bot commented Nov 17, 2025 •

edited

Loading

Uh oh!

dosubot bot commented Dec 15, 2025 •

edited

Loading

Uh oh!

PeterStaar-IBM left a comment

Uh oh!

vagenas left a comment •

edited

Loading

Uh oh!

ceberam commented Jan 26, 2026

Uh oh!

cau-git left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

ceberam commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Protections

🟢 Enforce conventional commit

🟢 Require two reviewer for test updates

Uh oh!

codecov bot commented Nov 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dosubot bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PeterStaar-IBM left a comment

Choose a reason for hiding this comment

Uh oh!

vagenas left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ceberam commented Jan 26, 2026

Uh oh!

cau-git left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ceberam commented Nov 17, 2025 •

edited

Loading

github-actions bot commented Nov 17, 2025 •

edited

Loading

mergify bot commented Nov 17, 2025 •

edited

Loading

codecov bot commented Nov 17, 2025 •

edited

Loading

dosubot bot commented Dec 15, 2025 •

edited

Loading

vagenas left a comment •

edited

Loading